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Abstract 

The quickness of large network dynamics is quantified by the length of transient paths, an 
analytically intractable measure. For discrete-time dynamics of networks of binary elements, we 
address this dilemma with a unified framework termed state concentration, defined as the exponent 
of the average number of i-step ancestors in state transition graphs. The state transition graph is 
defined by nodes corresponding to network states and directed links corresponding to transitions. 
Using this exponent, we interrogate random Boolean and majority vote networks. We find that 
extremely sparse Boolean networks and majority vote networks with arbitrary density achieve 
quickness, owing in part to long-tailed indegree distributions. As a corollary, only dense majority 
vote networks can achieve both quickness and robustness. 
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I. INTRODUCTION 



Networks of binary elements are useful tools for investigating a plethora of dynamical 
behavior and information processing in biological and social systems. For example, various 
models of associative memory are used to study neural information processing jl-3|. Random 
Boolean networks, also known as the Kauffman nets, show rich dynamics and are used to 
model gene regulation |4 - 6 1 . Random majority vote networks are often used to understand 

nnn n 

mechanisms for ordering in neural information processing; [3|, [ZH9[, gene regulation [6|, and 
collective opinion formation in social systems 10) . 

Properties desirable for dynamics of networks of such binary units include robustness and 
quickness. A system is defined to be robust when flipping a small number of units' states 
does not eventually alter behavior of the entire network. For the random Boolean networks, 
;he robustness has been quantified in the context of damage spreading in cellular automata 
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Dynamics are usually called quick if an orbit starting from an arbitrary state reaches 
the corresponding attractor within a small number of steps on average, i.e., with a short 
transient length of the dynamics. It is known that the random Boolean networks show 
long-tailed distributions of the transient length, implying that the transient is long on av- 



erage 



15]. However, analytical evaluation of the transient length is difficult. Therefore, we 



theoretically study the quickness of dynamics by introducing a new concept of state con- 
centration, extending the previous statistical dynamical framework 7H9|. In particular, the 
exponent of concentration, which we introduce later, is an analytically tractable quantity 
to measure the quickness of dynamics in the random Boolean and majority vote networks. 
Using this exponent, we investigate the compatibility of the robustness and quickness in 
these two types of networks. 

For this purpose, we distinguish densely connected Boolean network (DBN), sparsely 
connected Boolean network (SBN), densely connected majority vote network (DMN), and 
sparsely connected majority vote network (SMN). We show that strong state concentration, 
accompanied by a power law type of the indegree distribution with exponential cutoff, occurs 
in the majority vote networks (DMN and SMN) but not for the Boolean networks except 
for extremely sparse cases. Then, we argue that the DMN is the only type among the four 
types of networks that realizes both robustness and quickness. 
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II. MODEL 



Let us consider a network of n binary units. We define discrete-time dynamics of the 
network by 

x i (t+l) = f i (x 1 (t),...,x n (t)) (l<z<n), (1) 

where Xi(t) G {1,-1} is the binary state of the ith unit at time t. For a random Boolean 
network, each fa is randomly and independently chosen one of the 2 2 " Boolean functions on 
the n units. For a majority vote network, 



fi(x u . . . , x n ) = sgn V" WijXj (1 < i < n), (2) 




where sgn indicates the sign function. We consider an ensemble of randomly generated 
majority vote networks where Wij are independently and identically distributed Gaussian 
random variables. In general, a constant or random threshold could be included in the 
above dynamical expression, which we omit here for simplicity. If the value of fa (x±, . . . , x n ) 
depends only on randomly chosen K units for each i, the model is called the -fT-sparse 
network jj, 5j. The DBN and DMN correspond to K oc n, and the SBN and SMN correspond 
to K n. We study typical dynamical behavior of the random DBN, SBN, DMN, and 
SMN. 



III. DISTANCE LAW IN STATE TRANSITION 

The network state at time t is given in vector form as 

x(t) = ( Xl (t),...,x n (t)). (3) 

Let X = {x} be the set of the N = 2 n states. Given a network, the state transition is a 
mapping from X to itself. We write it briefly as x(t + 1) = fx(t). 

The dynamics of the distance between two state trajectories has been studied to char- 
acterize dynamics in these networks. We define the normalized Hamming distance between 
two states x and y in X by 

1 n 

D(x,y) = —^2\xi- yi\. (4) 
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It should be noted that the distance is restricted to the range < D(x, y) < 1. \i d! = (p(d) 
holds true for a function (p(d) for any x and y almost always as n — > oo, where d = D(x,y) 
and d! = D (fx, fy), we call it the distance law. For the DBN, y?(d) — (d — 0) and 
ip(d) = 1/2 (d ^ 0). For the SBN (l6[ , 



For the DMN 



^(d) = (l/2) [l-Cl-d)*] 



y>(d) = (2/vr)sin- 1 \/d. 



For the SMN LL 



-d) 



(5) 



(6) 



(7) 



where ( . ) is the binomial coefficient and 



^ J = (2/7r)sin- 1 v / j7^. 



(8) 



The dynamics of the distance under annealed approximation are given by d t+ \ = (p (dt) 
Q, fl^l"!^!]. For all the four types of networks, ip(0) = 0. For the DBN, <p(d) is discontinuous 
at d = 0, and the dynamics are essentially the same as those of the random mapping on 
the N states. Therefore, various properties of dynamics such as the number of attractors, 
transient length, and cycle period are well characterized . For the SBN, DMN, and 

SMN, tp(d) is continuous at d = 0. It is known that d = is a stable fixed point of mapping 
tp (i.e., < (p{0)' < 1) only for the SBN and SMN with K = 1 or 2. Otherwise, dt converges 
to a positive value d satisfying d = </? (dj . The convergence of the distance is usually fast 
and happens after ~ 10 steps except for DBN where one step is enough for the distance to 
converge. 



IV. EXPONENT OF STATE CONCENTRATION 



The transient length before the orbit enters the attractor is analytically intractable. To 
quantify the quickness of the dynamics by using an alternative order parameter called the 



exponent of state concentration, we use the so-called state transition graph |4|,|5|, [19] defined 
as follows. A map /, either Boolean or majority vote, induces a graph on N nodes. Each 
state x e X defines a node and has exactly one outgoing link directed to node fx. 



Suppose that each of the N = 2 n nodes (i.e., states) has a token at t = 0. For each t(> 0), 
an application of / moves all the tokens at each node x to node fx. Repeated applications 
of mapping / elicit concentration of tokens at specific nodes. We denote by f~~ l x the set 
of nodes whose tokens move to x after t steps and by \f~ l x\ the number of tokens at node 
x after t steps. Tokens are initially equally distributed, i.e., |/°cc| = 1 for each x, and the 
total number of tokens is conserved, i.e., 



El/ 



-t 



x 



N 



(9) 



for t > 0. 

The indegree of node x in the state transition graph is equal to \f~ l x\. The nodes with 
f _1 x = 0, where is the empty set, do not have parent nodes. The set of such nodes 
is called the garden of Eden |4|. and denoted by E\. The nodes x E E\ only appear as 
initial state. Only the nodes x e X — E\ receive tokens at t — 1. In general, we define 
E t = {x | f~ l x = 0}, i.e., the set of nodes that do not own tokens at time step t. There 
exists integer T such that 



C Ei C E 2 C ■ ■ ■ C E 7 



E- 



T+l 



E* 



(10) 



where T is the longest transient period and E* is the set of the transient states. The set of 
the attractors is given by A = X — E* (Fig. [T]). 




FIG. 1: Schematic of dynamics of state concentration. E\ indicates the garden of Eden, E 2 
indicates the nodes that do not have grandparents, and A indicates the set of attractors. 



To quantify the state concentration, we consider the frequency with which a randomly 
selected token at £c(0) and t = meets other tokens after applying /. We write the rela- 
tionship fx(0) = jc(1) succinctly as x(0) — > x(l). Let S\ be the conditional expectation 
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E [|/ _1 cc(l)| | x(0) — > x(l)] of the indegree of node £c(l) given that x(0) — > x(l) and that 
£c(0) is selected with equal probability (i.e., l/N). In general, we denote by S t the expected 
number of t-fold parent nodes of a node x(t) conditioned by a state transition path ending 
at x(t) through which a token has traveled, i.e., 

S t = E [|/ _t a;(t)| | aj(0) ->■ > x(t - 1) -> ac(t)] . (11) 

Obviously So = 1 and sequence {S t } is monotonically nondecreasing in t. If 

S t = e c ^, a > 0, (12) 

holds true for large n, the tokens are exponentially concentrated on nodes having at least a 
t-fold parent node. We refer to 

q = lim — - — - (13) 

as t-step exponent of concentration. Exponent q provides an n-independent measure of 
state concentration without explicitly evaluating the statistics of the transient. 

The stochastic symmetry of units makes the calculation of St tractable. To explain 
the symmetry, we consider the majority vote network; similar arguments hold true for the 
Boolean network. Because weights are independently and identically distributed, the 
probability distribution of mapping / is invariant under permutation of x\, . . ., x n , which 
are passed as the arguments to fi (1 < i < n). In addition, the probability is invariant 
under the flip of sign of each x\. Therefore, the following gauge invariance holds. First, the 
probability distribution of x(t) is invariant under permutation of the unit indices. Second, 
the probability distribution is invariant under the sign flip of any Xi(t). Because any state 
is mapped in a single step to a given x by permutation and sign flip, all the states are 
stochastically equivalent. Therefore, Prob{|/ _1 a3| = k}, for example, is the same for all x, 
and Prob{cc(0) — > x(l)} = l/N for any je(0) and x(l). 

We define 

r k = Prob {|/ -1 a;(l)| = k | x(0) -> sc(l)} , (14) 

i.e., the indegree distribution of node x(l) conditioned by x(0) — > x(l). The symmetry 
guarantees that r k is independent of x(0) and x(l). Let us compute 

N 



Sl = J2kr k . (15) 



k=i 



We denote by 2/(0) a node such that D (x(0), 2/(0)) = d for a given x(0). The number of 
nodes with distance d away from cc(0) is given by 



M « e^W, (16) 
where 

#(d) = -dlogd-(l-d)log(l-d) (17) 

is the entropy. The probability that D (fx(0), fy(0)) = d' (see Fig. [2] for a schematic 
illustration of this situation) is given by 

^(d!\d) = {^)ip(d)^[i-ip(d)]^. (is) 

In particular, 

0(0 | d) = [1 - ^(d)] n = Prob {/2/(0) = sc(l) | x(0) -> sb(1)} . (19) 
By using the saddle-point approximation, we obtain 

n 

^ = EG*(oi<o 

nd=Q 

« y exp n {if (z) + log [1 - p(z)] } oc e nci , (20) 



where 



and 



Cl = if (<f) + log [l-^(cf)] (21) 



cf = arg max {if (d) + log [1 - <p(d)] } . (22) 

To evaluate q in general, we consider a t-step state transition path X t = {x(0) — > x(l) — > 
• ••—>■ x (t) — x*} ending at x* and calculate the conditional probability that another path 
Y t — {2/(0) —> ■ ■ ■ — > y(t)} ends at the same x*. S t is the expectation of the number of such 
t-step paths. Let us denote the distance D(x(t'),y(t')) by d t /, where < t' < t and d t = 0. 
Then, under the Markov assumption, the probability of path Y t conditioned by path X t is 
represented in terms of the distances of the two sequences, i.e., df, < t' < t, by 

t'=t 

Prob{Y t \X t } = n^K'l^-i)- (23) 
t'=i 
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FIG. 2: Schematic of one-step dynamics of the distance. 



The Markov assumption is valid for large n and a finite t. Because there are states y(0) 

possessing distance do from x(0), the expected number of paths is given by the integration 
of J Prob {Y t \ X t } over all the possible distance sequences {do, ■ ■ ■ , rft-i, d t = 0}. By 
using the saddle-point approximation, we have 



S t = exp 



nH{dl) + Ys lo ^ d t'\ d *t'-i) 

t'=i 



(24) 



for large n, where d*, are the maximizers of the integrand in the path integration. Equa- 
tion (B1J) implies 



ct=x;w^-i)+^iog^-i) 
f=i 

+(l-dj) log [1-^-0]}. (25) 
For example, for t = 2, we obtain 

c 2 =/d(d*) + if (dj) + log [l-<p (dl)} + d\ \og V (dl) 

+ (1 - dl) log [1 - <p {dl)] , (26) 

where 

{d* , d\} = arg max {H(d Q ) + H(di) + log [1 - ip (d x )] 

+ di log ^ (d ) + (1 - di) log [1 - p (d )]} . (27) 

On the basis of the expression of tp for the SBN and SMN shown before, the dependence 
of ci, C2, C3, and C4 on fT is plotted in Fig. El For the SBN, c t (1 < £ < 4) converges to 
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quickly as K increases. For the DBN, which is the case for K = n, we trivially obtain c t = 
at least for small t because / is equivalent to the random mapping. Figure [3] indicates that 
the state concentration occurs only for very small K in the random Boolean network. In 
contrast, the state concentration occurs even for large K in the majority vote network. In 
particular, for the DMN with K — > n, we obtain c\ ~ 0.157 js]. Figure |3j also indicates that 
the state concentration quickly proceeds as t increases, except in the DBN. 

0.6 
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t=2:SMN 
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K 

FIG. 3: (Color online) c\, C2, C3, and C4 for the SMN and SBN. 



We verified Eq. (fT5|) by comparing St obtained from direct numerical simulations (i.e., 
Eq. (|lip ) and c t generally given by Eq. fT25]) . The results shown in Fig. H] indicate that the 
theory (lines) seems to agree with numerical results at least for large N; although the largest 
N value shown in the figure is only n = 25. Therefore, the Markov assumption (Eq. (T23l) ). 
also implicitly assumed for t — 2, 3, and 4 in Fig. [3J, roughly holds true up to t ~ 4 for large 
n. 

Theoretically, most sequences Y t that meet X t after t steps of state transition own the 
sequence of distance given by d* t = {0^, d*,--- , — 0}. In particular, a majority of 

the initial states Y is initially separated from X by d^t). Figure H] suggests that this is 
the case at least up to t ~ 4 for large n. The sequence of distance is shown for 1 < t < 4 
in Fig. [51 
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10 3 10 4 10 5 , T 10 6 1 7 1 8 1 3 1 4 10 5 , T 10 6 10 7 10 : 

N N 



FIG. 4: (Color online) St exponentially increases with n except in the DMN. (a) DBN, (b) SBN 
with K = 3, (c) DMN, and (d) SMN with K = 3. The lines in (b) indicate the theoretical estimates, 
i.e., 5i oc ]V - 321 , £ 2 K ^ K and 5 4 K ^o.eoo^ Similarly, the lines in (c) indicate 

Si OC jV ' 226 , 5 2 K £ g K ^0.324 ^ and ^ K ^yO.SSS^ Thg lines in ( d ) indicate ^ K ^0.370 ^ 

(S*2 oc iV 0,523 , S3 oc iV 0,608 , and S4 oc jV 0,663 . Each point in the figure represents the average of St 
over 10 3 realizations of the network. 



V. INDEGREE DISTRIBUTION OF THE STATE TRANSITION GRAPH 

We calculate the incoming degree distribution 

p k = Prob {|/ -1 a;| = k] , (28) 

where k is the indegree of a state and J2k=oP k = 1- Because each node in the state transition 
graph has exactly one outgoing link, we have 

N 

(k)=^2kp k = l, (29) 

k=0 
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FIG. 5: (Color online) Sequence of the most likely distances = {d,Q,d*,- 
1< t < 4. 



,dU,d$ =0} for 



where (•) indicates the expectation. Because r k = kp k [7[ (also see 
obtain 

N N 



20| for an example), we 



(30) 



k=0 k=0 

Therefore, c\ > indicates that (k 2 ) diverges in the limit of N — 2 n — > oo, reminiscent of 



2fJ-|22|. 



the scale-free property of the state transition graph 

For the DBN, the state transition graph is the directed random graph in which p k obeys 
the Poisson distribution (i.e., p k = 1/ekl) with mean and variance 1 jj, Therefore, 
(k 2 ) = 2, proving that c\ — for the DBN (i.e., no exponential state concentration). This 
is consistent with Fig. |4](a) (circles). Figure [3] suggests that c\ ~ when K is approxi- 
mately larger than 10. Therefore, the degree distribution of the state transition graph is 
also narrowly distributed for the SBN with K > 10. We verified that the numerically ob- 
tained indegree distribution for the random Boolean network with n = 30 and K = 20 
approximately obeys the Poisson distribution (Fig. E^a)). 

In contrast, a positive value of c± found for the SBN with small K, DMN, and SMN 
(Fig. [3]) indicates that (k 2 ) (= Si) diverges exponentially in n. This is actually the case, 
as shown in Fig. HJ^b, c, d). For scale-free networks with p k rs fc -7 , the extremal criterion 



would lead to 7 ~ (ci + 31og2)/(ci + log 2) [2l|, |22|. However, the degree distribution 
numerically obtained for the DMN, shown in Fig. EJ^b), deviates from a power law. The 
degree distribution numerically obtained for the SBN is also different from a power law 19]. 
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To guides to the eyes, a fitting curve on the basis of a power law with an exponential cutoff 
is shown by the line in Fig. Efb). In fact, the power law is not the only distribution that 
yields the divergence of (k 2 ). In the present case, the position of the exponential cutoff may 
mildly diverge as N becomes large. 




k k 
FIG. 6: (Color online) Complementary indegree distribution of (i.e., = Ylk>=kPk') of the state 

transition graph, (a) Random Boolean network with n = 30 and K = 20. The numerical results are 

shown by squares, and the Poisson distribution with mean 1 is indicated by the line, (b) Random 

majority vote network with n = 30 and K = 30. A fitting curve, P^ oc k~ 1,72 exp {— 0.001/c}, 

obtained from the least square error method, is shown by the line as guides to the eyes. 

VI. DISCUSSION AND CONCLUSIONS 

In summary, we provided a unified framework for analyzing the state concentration. We 
found that the state concentration occurs in the SBN with small K, DMN, and SMN, but 
not in the DBN. We also revealed the long-tailed distributions of the indegree of the state 
transition graph in the SBN, DMN, and SMN, but not in the DBN. 

We briefly discuss the relationship between the quickness, measured by the exponent of 
state concentration in this study, and the robustness of the dynamics. The robustness of 
the dynamics is often measured in terms of damage spreading. It is a long-term property 
concerning the stability of d = for mapping </?. As we mentioned, d = is an unstable fixed 
point of tp unless K = 1 or 2. Although the SMN with K = 1 or 2 satisfies quickness and 
robustness, we do not discuss these cases because the dynamics in these cases are just frozen 
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18] . Here we consider a simpler measure of robustness based on a one-step property, 



i.e., how a difference by a single bit evolves after a single application of /. This is essentially 
the same as the Boolean derivative, a measure of the robustness used for analyzing random 



Boolean networks 



11 



14j . In the SMN, the probability that a single bit flip results in a 



single bit flip after the application of / is given by 



g K>1 = (2/tt) sin- 1 y/l/K. (31) 



Because gx,i is equal to 0.5 for K = 2 and decreases according to ~ 2/(ttvK) as K 
increases, the random majority vote network is robust except for very small K. However, 
in the random Boolean network, the same flip probability is equal to 1/2 — 1/2 2A , which 
quickly approaches 1/2 as K increases. In particular, the flip probability for the Boolean 
network at K = 2 is equal to 7/16 = 0.4375, which is close to those for the majority vote 
network with K = 2 (i.e., 0.5) and K = 3 (i.e., 0.392). Although the Boolean network with 



K = 1 has sufficient robustness, the dynamics in this case are frozen [23|. Therefore, we 
regard that the robustness of the one-step dynamics holds true in the DMN, but not in the 
DBN, SBN, and SMN. By combining this observation with our main results, we regard that 
quickness and robustness suitably compromise in the DMN, but not in the DBN, SBN, and 
SMN. 
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